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Abstract 

Life history of organisms is exposed to uncertainty generated by internal and external stochasticities. Internal stochasticity is 
generated by the randomness in each individual life history, such as randomness in food intake, genetic character and size 
growth rate, whereas external stochasticity is due to the environment. For instance, it is known that the external 
stochasticity tends to affect population growth rate negatively. It has been shown in a recent theoretical study using path- 
integral formulation in structured linear demographic models that internal stochasticity can affect population growth rate 
positively or negatively. However, internal stochasticity has not been the main subject of researches. Taking account of 
effect of internal stochasticity on the population growth rate, the fittest organism has the optimal control of life history 
affected by the stochasticity in the habitat. The study of this control is known as the optimal life schedule problems. In order 
to analyze the optimal control under internal stochasticity, we need to make use of "Stochastic Control Theory" in the 
optimal life schedule problem. There is, however, no such kind of theory unifying optimal life history and internal 
stochasticity. This study focuses on an extension of optimal life schedule problems to unify control theory of internal 
stochasticity into linear demographic models. First, we show the relationship between the general age-states linear 
demographic models and the stochastic control theory via several mathematical formulations, such as path-integral, 
integral equation, and transition matrix. Secondly, we apply our theory to a two-resource utilization model for two different 
breeding systems: semelparity and iteroparity. Finally, we show that the diversity of resources is important for species in a 
case. Our study shows that this unification theory can address risk hedges of life history in general age-states linear 
demographic models. 
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Introduction 

Environmental stochasticity is one of the problems which 
organisms face in their life schedule because it brings uncertainty 
to their maturity and reproduction timing. Demographers 
examine the effect of stochasticity on population dynamics by 
using linear demographic models (LDM), such as transition matrix 
models (TMM), integral projection models (IPM), and partial 
differential equations (PDE). As a consequence, they showed that 
stochasticity affects population growth rate negatively [1-6]. 
Empirical researchers are devoted to estimating the effect of 
external stochasticity on the life history of organisms [7—9]. 
However, there actually exists a twofold stochasticity in their 
models, that is, internal and environmental stochasticity (i.e., 
external stochasticities). For example, in TMM used by ecologists, 
the internal stochasticity yields a set of transition probabilities to 
other states and is generated by the dispersion of each individual 
life history, such as the dispersion of feed intake, genetic character, 
and size growth rate; whereas external stochasticity annually 
modifies the value of these transition probabilities, ft is difficult to 
distinguish one stochastic effect from the other. Research on 
internal stochasticity is limited because the effect is different 
among formalizations of LDMs and we have not had the proper 



methods to analyze the effect of internal stochasticity on the 
dominant eiganvalue. 

In a recent study, Oizumi and Takada (2013) obtained the 
methodology to show that internal stochasticity could increase and 
decrease population growth rates depending on the breeding 
system [10]. Additionally, the study provided a way to analyze the 
effect of the internal stochasticity on the population growth rate by 
using formulae of stochastic differential equations. Life history 
with internal stochasticity generates a population density function 
(population vector) that can be expressed by an age-size structured 
LDM. Then, the effect of internal stochasticity appears as a 
diffusion term in the LDM. The dominant eigenvalue of LDM is 
called intrinsic rate of natural increase, or fitness (which this paper 
uses) in theoretical demography, and is derived from the 
characteristic equation of the LDM, so called Euler-Lotka 
equation. The equation is composed of the breeding system, 
mortality, and the physiological growth process affected by 
internal stochasticity. In nature, it is considered that species 
control those three elements to become optimal in the habitat. 
Finding the optimal control is called "optimal life schedule 
problem" (OLSP) in theoretical biology. The analysis of LDM and 
OLSP was unified by Taylor et.al (1974) and Leon (1976) for 
deterministic life history via methods of dynamical programming 
[11,12]. Oizumi and Takada (2013) extended the proof to a life 
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history with stochastic growth via a novel method that is called 
path-integral formulation; besides, they showed by their method 
that internal stochasticity serves as a significant factor involved in 
persistence of species as well as external stochasticity [10]. 
Considering control of the stochasticity is hence concerned with 
not only various issues on conservation ecology, but evolution of 
life history. Internal stochasticity has a lot of ground to cover as 
mentioned above, even though their theory deal with optimal 
breeding timing in life history affected by an internal stochasticity 
only. To take account of multifaceted parts in life history, their 
theory should extend to multi-stage LDMs affected by several 
numbers of internal stochasticities and be able to analyze not only 
optimal control of reproductive timing but also of each transition 
rate. In this paper, we focus on unifying "Stochastic Control 
Theory" into multi-stage LDMs for that reason. This theory is 
extension of dynamical programming to control of stochastic 
process, i.e. we examine the theorem of Taylor et.al (1974), Leon 
(1976), Oizumi and Takada (2013) to adapt not only optimal 
reproductive timing but also the whole control of life history with 
internal stochasticity. 

Stochastic control theory is normally used in control engineer- 
ing and mathematical finance. In mathematical finance, it is 
applied to the analysis of optimal risk management in personal 
assets, where the optimal control determines utilization of 
investment between investment trust and loss insurance for the 
personal utility maximization [13,14]. Mathematical economists 
define a function, called utility function, and search for the optimal 
control maximizing it. The function normally belongs to a class of 
concave functions and is assumed by it. Their convexity is a key 
point finding the optimal strategy because this property provides 
some extreme values related to the strategy. The idea of 
economists resembles that of an organism utilizing resources for 
its own maintenance and reproduction. In fact, mathematical 
biologists studying OLSP use functions, called reproductive value, 
similar to utility functions used by economists [15,16]. Those 
functions are generally called objective functions in control theory. 
To unify OLSP with internal stochasticity into LDMs, an 
optimized objective function in biology must yield the maximum 
fitness. However, reproductive values which appear in their 
analysis are not always clear relation with the fitness. Studies of 
Taylor et.al (1974) and Leon (1976) postulated only age- 
structured LDM and is obscure in LDMs incident to their 
objective function, respectively. Under internal stochasticity, 
Oizumi and Takada (2013) asserted that the objective function 
should be the Laplace transform of the expectation of the 
reproductive success (ERS), because it is generated by the 
Euler— Lotka equation in the age-size structured model. The 
function has the same meaning as the one used in Taylor et.al and 
Leon. Although Oizumi and Takada mentioned the relationship 
between path-integral formulation and other LDMs, they were not 
enough to elucidate the correspondence to all formulation of 
LDMs. 

In this paper, we address control of a life history having rf-states 
affected by N internal stochasticities as extension of the previous 
work [10]: besides, we show the relationship between path- 
integral formulation and other LDMs in the life history. We unify 
OLSP and LDMs at the life history via an optimized objective 
function derived from an important equation from control theory 
(Hamilton^Jacobi-Bellman equation). Applying our theory to 
analysis of a simple risk control problem (two-resource utilization 
model) in two distinct breeding systems, we discuss the meaning of 
the maximized objective function and the relationship between the 
optimal strategy and the structure of population. 



Analysis 

Configuration of individual life history 

Several stochasticities in life history have been assumed by 
diffusion process such as migration of population [17,18], size 
growth [10,19], personal assets in human society as well [13,20]. 
We assume to categorize them with effect of internal stochasticity 
and construct the configuration of two types of breeding systems 
(semelparity and iteroparity) as general stochastic control process 
in this section. 

States transition processes. We consider individuals hav- 
ing (/-state X a eA^U d (d>\) at age a, A being the domain of 
each state. They are assumed to be fluctuated by the N internal 
stochastisities. Then, the growth rate of each state is provided by 
the following Ito's SDE: 



dX>= gj (X a ,v a )da + Y,Ll°jk(Xa,Va)d& a 1 <j<d 



(1) 



where x J (l<j<d) represents an initial state ofy'-th state, X 1 . In 
this study, we assume all individuals have the same inital state x. 
On the right-hand side, the first drift term represents deterministic 
rule of growth process of the element, whereas the second term 
represents fluctuation at X J g , and B k a denotes an element of TV- 
dimensional Brownian motion. We set 

v=v a : =(vi(a),V2(a),---,v;(a))eV<=R' (/>1) 

which represents a control vector and V is a compact convex set of 
R'. 

Fertility function and breeding systems. We introduce a 
general fertility which F : ^4— >(0,co) has integrability with respect 
to a probability measure P(yedy) as follows: 



F(y)P(yedy)<co. 



(2) 



Semelparous and iteroparous breeding systems are defined as 
types of fertility function. 

In semelparous species, denoting mature state D^dAeA where 
8A is boundary of A and the mature age a* such that 

X a *eD, «* : =inf{ae(0,oo]|X a eZ>}, (3) 

a 

the fertility, F^(y) 3 is defined by 



F s (y) 



U(y) yeD 
\ 0 y<£D 



(4) 



where the fertility rate function satisfies <j>(y) > 1 at a mature state 
yeD. From biological point of view, x should be x^D. While, we 
define an iteroparous fertility function F\(y) as continuous, and 
measurable function given by Eq.(2). Then, the species can 
reproduce at any time during their growth state. 

We use fertility, Eq.(2), as a general breeding system if the 
analysis is common to semelparous and iteroparous breeding 
systems. 

Mortality and survivorship. Semelparous and iteroparous 
species have different survivorship. We set a common mortality 
function, fi : U d x R'->R + , depending on y as follows: 
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0<Ho<fi{y,v), 

in both types of species. A general survivorship S(a) until age a is 
written as 



S(a) = exp 



dxji(X z ,v x ) 



(5) 



Since semelparous species die upon reproduction, we can write 
the survivorship as 



S s (a) : = l{ a ^a*]}S(a), 



(6) 



by using an indicator function and the general survivorship, Eq.(5). 
While, we assume that iteroparous species have a limit of lifespan. 
Letting a < go be the limit of lifespan, the iteroparous survivorship 
is written as 



Si (a) : = l{ ae [o,a)}S(<; 



(7) 



If the maximum lifespan follows a = oo , the species only die by 
accidental death and Eq.(7) corresponds to Eq.(5). 

Difference of life history between semelparity and 
iteroparity. We here call Eqs.(l), (2), and (5) as life history. 
Then, we define a semelparous and an iteroparous life history 
following Eqs.(4) and (6), and F\(y) and Eq.(7) under the growth 
rates Eq.(l), respectively. In other words, we identify each life 
history with their difference in fertility and survivorship. 

LDM and objective function 

In order to analyze that two life histories influence each 
population dynamics, we unify the individual life history into its 
population density function formalized by path-integral: after that, 
we demonstrate similarities among LDMs via the path-integral 
formulation in this section. In this study, we refer to this sort of 
population density function parametarized by time, age, and states 
as "population vector". 

Age-state structured model with internal 
stochasticity. Setting x : = (x l ,x 2 , ■ ■ ■ ,x' , ■ ■ ■ ,x d ) as an initial 
state vector of Eq.(l), we consider the following quantity: 



u a (x) : = E x [F(X a )S(a) 



(8) 



where this function represents the expectation of the product of 
fertility and mortality at age a, and we refer to it as an expectation 
of reproductive success (ERS) wich is identical to net reproduction 
function in demography [21]. The expectation in Eq.(8) is given by 
the probability measure which Eq.(l) generates. Oizumi and 
Takada showed, in d = 1 and N= 1, that a characteristic equation, 

M*) = L ( 9 ) 

is composed of the Laplace transform of Eq.(8), which is 



da exp{ — la}u a (x) , 



(10) 



and provides the fitness X* of age-size structured model [10]. 
According to this consequence, we then can expect to find a 
general age-states structured population model composed of the 



life history, Eqs.(l), (2), and (5). From Feynman-Kac formula (e.g., 
SI in File SI), ERS satisfies the following equation: 



8 

da 



u a (x)=-H v x ii a (x) 



d 1 



J J = 1 

N 



(11) 



Cj/(x,v) : = ^o jk (x,v)cj k (x,v) 



. uo(x)=F(x) 



Setting Ka(x— *y) represents a transition rate (including survivor- 
ship) from initial state x to final state y at age a (projection 
function), ERS can be written by 

E x [F(X a )S(a)]= ^dyF(y)K a (x^y), for F(y)eC 2 0 . 

It is then known that there is a relationship between tpeC 2 and K 
from partial integration, such that 



dyq>(y) —K a {x^y) ■■ 



dyH v My)K a (x^y) 



dy<p(y)HlK a (x^>y), 



(12) 



where T~C y denotes the adjoint Hamiltonian of H" y (Fokker-Planck 
Hamiltonian), 



(13) 



From the arbitrariness of f, the projection function K satisfies the 
following Fokker-Planck equation: 



da 



K a (x^y) = -H v v K a (x->y) 



UmK a (x^yy) =6 (x—y), 

a|0 



where <5 (x—y) is (/-dimensional Dirac's delta 



(14) 



a 

(x-y):=Tld(xJ-y), 



This consequence suggests that the dominant characteristic root of 
Eq.(10) and the projection function, K a (x—>y), can compose the 
population vector, P t (a,x->y), at time t and age a with transition 
from x to y satisfying the following a stable demographic model, 



'3 3' 

dt da 
P,{0,x->y)=n t (x)d"(x~v) 
P 0 (a,x,y)=V(a,x,y), 



P,(a,x->y) = -H y y P t (a,x->y) 

( 15 ) 



because the dynamics of Eq.(l 1) is provided by the Fokker-Planck 
operator generating the cohort dynamics [10]. V(a,x,y) is 
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assumed by 

V(0,x,y)=V(x)d d (x-y) (16) 
The integral equation of this vector in offspring dynamics is 



n,(x) 



da 



dyF(y)P,(a,x->y), 



(17) 



where n t {x) and Hq(x) represent offspring number at time t and 
the initial number of offspring, respectively. In comparison with 
Oizumi and Takada's work [10], the population vector should be 
decomposed into the initial offspring number and projection 
function, such that 



n t - a {x)K a (x->y) 



t — a>0 



P,(a,x->y)-- 



d£ J Ua (^y)V(a-t,x£) a-t>0. 



(18) 



In a>t, the projection fuction represents projection of states from 

a — t to a and let it satisfy lim,|o Jt,a{^y) = S d {^ — y). Eq.(18) is 
generalization of boundary condition with respect to age a in 
continuous age structured models and we describe their expres- 
sions later [22]. The above decomposition can be proved in more 
general configuration (in non-autonomous system) of the life 
history by the formal solution of Eq.(15) (e.g., SI in File SI). 
Setting the inner product 



Eq.(l), this function have translational symmetry with respect to 
age a because of strong Markov property in the SDE as follows: 



(21) 



The path integral is a summation over an infinity of possible 
growth curves connecting x with y with the sieve of mortality to 
compute the density in Eq.(5). Eq.(19) is known in physics as the 
Hamiltonian expression. The algebric form of Fokker-Planck 
Hamiltonian, Ti >T ( — iq t ,X T ), actually appears in the action 
integral of this expression. Substituting Eq.(18) into Eq.(17), we 
obtain a renewal equation of offspring dynamics as follows: 



n,(x) 
G,(x) : 



-G,(x) 



da n t _ a (x)u a (x) 



| cfaj d(V(a-t,x,Z)u,(Z) (22) 
da d£ dyV(a- t,x,£)F(y)J t<a (£->y). 



Since the ERS is always nonnegative and bounded for all age a 
(liniaioo M a (x)-»0) in biological assumption (Eqs.(2) and (5)), 
Eq.(22) corresponds with the mathematical form of the classical 
renewal equation which appears in McKendric equation [21]. 
Therefore, we can adopt Feller's methods [23] to solve Eq.(22) 
because this equation satisfies all conditions to use Sharp-Lofka- 
Feller theorem [21]. The population vector, Eq.(18), then is 



-iq 1 -X x =-iJ2q i T X J z , 

7=1 

the projection function by a path-integral formalization becomes 



Ji 0 =i Jr" [Jo \(-iq,,X x ) 

\ X " 'v(x)\ V{q) : =[••• f I II dxA- \ II dq, 

Sx Q =x W J L z «(<>,«) J jVre(0,«) ( 19 ) 

d 1 d i 

H'<(-iq z ,X,) : =-J2rtgi( X ^)+?Yl ^C j(X x ,v,) + h(X z ,v x ) 

jj-l 

K 0 {x^y)=d d {x-y), 



where X a , q T eU d , and Z denote the formal differential of X a with 

respect to a, the adjoint vector, and the normalization factor of 
transition probability which gives 



^dyK a (x^y) = E x [S(a) 



respectively. / (>a (£— >.y) denotes the projection function in t<a 
and is defined by 



x a =y 



x a -t = i 



V(x) 



, V{q) exp 



dz(- iq x -X x -TC T (-iq x ,X x )) | . 

(20) 

When transition of all states follows autonomous system, such as 



P,(a,x->y) = 

C x ex V {A*{t-a)}[l + 0(exp{-r,{t-a)})]K a {x^y) 
| from Sharp — Lotka — Feller theorem (i?>0) 



C x : = 



dtexp{-)*t}G,(x) 
0 >0. 



d_ 
dX 



(23) 



Then, )* is provided by the dominant characteristic root of Eq.(9) 
(e.g., P.80 at [10]). Consequently, ty\(x) satisfies the theorem 
proved by Taylor et. al (1974) and Leon (1976) because the 
function is monotonically decreasing in X [11,12]. The theorem 
shows that a strategy maximizing if)\(x) is equivalent to 
maximizing the fitness, )* . Therefore, we can adopt )\l\{x) as 
objective function in optimal life schedule with stochastic 
development. Hereafter, we call ^)\{x) "objective function". 

Correspondence of path-integral formulation to other 
LDMs. In this part, we show the correspondence of Hamilto- 
nian and projection function in our formalization to elements and 
parameters in the other formalization of LDMs: IPM, PDE, and 
TMM. Mathematical correspondence among those three models 
is introduced in some text books once in a while, such as [1,24]. 
For instance, Takada and Hara provided an example that 
elements of a size structured TMM (Lefkovitch matrix) can 
generate size structured PDE (Fokker-Planck equation) at 
d=\,N= \ [25]. 

In our formalization, the projection function composing the 
solution of Eq.(15) can serve as the following IPM because of its 
Markov property [10]: 



P t+e (a+e,x^y) = j di,K e {^y)P,{a,x^) 



(24) 
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On the other hand, We can show an age-size TMM corresponding 
to Eq.(22) at d = 1 and N= 1. Let 

a: = (0,1,2, a) 

y : ={yo,y\,y2,- ■ ■ ,y m ,- ■ ■ ,yjw) 

and o) be the age vector, the size-vector, and co = ix(M+ 1) + 1, 
respectively. Setting a compact set ^4elR + and size interval h 
deviding A into M categories, y m represents m-th size category. 
When P ;+ 1 (a,y) denotes a population vector in the TMM, we can 
show the following w x w TMM: 



P m (a,y)=LP ( (a,y), 



(25) 



where 



L = 



"/o 


F 


... f 


F" 


K 0 


0 


... o 


... o 


0 


Ki 


... o 


... o 




0 


'•• 0 


... o 


0 




0 ••• 


K«_i 0 



(26) 



is a candidate of the discretized Eq.(24) in d= 1 and N= 1. This 
TMM is common with Leslie matrix on the mathematical form, 
however, each element (vital rate) is composed of the following 
matrices: F, O, and K a are F a = (/ 0 ••• f m ■■■ /m), 
(M+l) x(M+l) zero matrix, and (M+ 1) x (M + 1) size 
transition matrix if and only if Ko : = (^o,o> ' ' ' J<-o,m, ' ' ' ^o,m) > 
respectively. 

Suppose that Y a is a stochastic process of the size growth 
generated by the following parameterized collection of random 
variables, 

{Ya} asm :={y a Ma)} ae[M , M0)=0). (27) 
The fitness follows the characteristic equation of L (e.g., S 1 in File S 1) 



l=J2E yo \f(Y a )S(a)]^ a - 



(28) 



This characteristic equation parallels Eq.(9) and represents the 
generalized Euler-Lotka equation in the TMM to age-size model. 
Thus, path-integral expression is hence one of formulation in LDMs. 
There are two reasons for adopting Hamiltonian expression of the 
path-integral; The first is easy to treat any cases even if gj(X a ,v a ) and 
Cjj'{X a ,v a ) are not differentiable and including zero, such as 
Gjk(X a ,v a ) =0, respectively and the other is of use to deal with the 
relationship among Hamiltonians involved in the states transition 
and the optimal schedule problem in the next section. 

HJB equations and analysis 

We suppose 



sup 

vieV J 



daexp{-la}E x [F(X a )S{a)], (29) 



from Eq.(10) in this section, because the theorem of optimal life 
schedule (e.g., S2 in File SI) shows that optimal strategy of life 



history is equivalent to finding a function ^x( x )- This function is 
called "value function" in control theory. We adapt Bellman's 
dynamical programming represented by PDE in stochastic control 
theory to find optimal control in both types of life history, and, 
address differences of the optimal control between semelparous 
and iteroparous species. This approach is not so famous method 
compared with "optimal life schedule problem" (OLSP) associated 
with "Maximum Principle (MP)". Although there are some 
reasons, precursors pointed out that Bellman's approach needs 
the value function having sufficient smoothness to use partial 
differential in the analysis (e.g., Pontryagin's classical text book 
[26]). On the other hand, the famous approach which is MP does 
not need this sort of smoothness. However, the situation is 
changed by the appearance of viscosity solution in 1980's [27-29]. 
That is an extension of solution in differential equation and can 
eliminate the smoothness from the necessary condition in Bell- 
man's approach. Moreover, it is enable for us to proof the 
equivalence relationship between both approaches via Hamiltoni- 
ans appeared in both analyses (e.g., S3 in File SI) [30,31]. Those 
two approaches have different biological points of view that 
Bellman's approach is to find optimal behavior as cohort dynamics 
represented by PDE, whereas MP is to find that as individual states 
transition represented by SDE. As mentioned previously, math- 
ematicians showed that those approaches produced the same 
consequence without their intention. In this section, we adapt 
known mathematical consequenses of Bellman's dynamical 
programming to the value function and derive general biological 
interpretations from that value functions in semelparous and 
iteroparous species are characterized by different equations. In the 
next subsection, we handle the relationship between the stochastic 
control theory involved in dynamical programming and OLSP. 

HJB equation and OLSP. In oreder to analyze the value 
function, Eq.(29) and derive the equation what it obeys, we 
introduce ERS with exp{ — la} 

wx,a{x) : = exp{-la}u a (x), (30) 
and a new value function described by age backward is; 



wx, a ( x ) : = sup{exp{ - A(a. -a)}E x {F(X x - a ) S(cc- a)}}, (31) 

veV 



to use the following relationship: 

hv, 0 (x) = sup jexp{ - l{a - a 0 )}E x \w^ a (x a - % ) S{a -a 0 )] }, 

(32) 

where 0<ao<a<a. This equation implies that value function 
controlled from «o to a corresponds with Wx,a 0 (x) by using Eq.(3 1) 
(known as Bellman principle). Note that Wx,o{x) represents the 
value function at age a in the original variable of a. Then, Eq.(29) 
is rewritten as 



&x,J x ) 



da Wx, a {x). 



(33) 



Due to Eq.(9), the population growth of the fittest species 
asymptotically follows the fitness, 1, which is characteristic root of 



Applying Feyman-Kac formula to Eq.(32), we have 



(34) 
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0= sup{exp{-l(a-o 0 )}E*[B'A,„(x a _ ao )s(a-ao)]}-M'j,a 0 W 

" eV ' (35) 

= sup{ i;- a " dr exp{-AM}E ;t [(| 1 * J , 00+r (X r )-[^ + A]^ ao+r (Z r ))s( r )] }, 

Cosidering a limit 

l i$l S ^{ J" * ex P{ -^•('■)}IE JC [(^«-^ +r (AV) - [WJ+A]iv^ +p (Jr,))s(r)] } =0 

(36) 

where h = a — ao, it provides an important equation (e.g., 
proposition and its proof on pp. 182-183 in [31]) that the value 
function holds as follows: 



inf{[W; + ^R a «}=0 



Wx A (x)=F(x) 



(37) 



This equation is known in control theory as Hamilton-Jacobi- 
Bellman equation (HJB equation) and essential of this study. When a 
control v* satisfies Eq.(37) or W^ a (x) exists under the control, v* is 
optimal [14,32]. Eq.(37) is, however, nonlinear and Wx, a { x ) generally 
does not have sufficient smoothness; the value function is interpreted 
as a "viscosity solution" in Eq.(37) (e.g., S4 in File SI and [30,31]). 

Since v* has a degree of freedom with respect to X and has to hold 
Eq.(34), the unique optimal control is given by v : = v*|^_j. Those 
two procedures are biologically essential because the optimal control 
depending on X means optimizing not only the ERS but also the 
speed of alternation of generations. In other words, to maximize the 
objective function is to maximize the ERS of precocious individuals of 
which has small reduction by exp{ — Xa}. Since w; >a (x) depends on 
age and states, it suggests that an optimal control, v(a,x), also 
depends on both state x and age a. Therefore, species having the 
optimal state transition at the original age, a, (age forward) becomes 



dX{ =gj(X a ,v(a-a,X a ))da+J2a hk (X a :v(a-a,X a ) )dB k a 

k=\ 

X 0 =x, ^ 



from Eq.(l). Due to Eq.(38) being non-autonomous system, the 
projection function requires more general form than Eq.(19) as 
follows: 



' K aG ( 0 ^(x->y) = 

" ' V(x) f £>(<7)expJ dz{-iq T -X t -n{x,-iq % J[^j[ 
H(i,-iq x ,X x ) : = 



(39) 



d d 

- £ iq[ gj{i,x T ) + \ <id x ijj{x,x,)+]i{x,x,) 



where 



gj{a,X a ) : =gj(X a ,v(a.-a,X a )) 
Cjj(a,X a ) : =c j (X a ,v(a-a,X a )) 
Ji(a,X a ) : =n(X a ,v(a-a,X a )). 



(40) 



This path-integral formulation admits a case that terms, gj(a,X a ^, 
<jj y k(a,Xa), and Jl{a,X a ) are not always differentiable everywhere. 

Stationary control problem of life history. If species die 
only by accidental death or programed death after their 
reproduction (such as semelparous life history or life history 
having sufficiently long lifespan), the case can adapt to their 
analysis, we can deal with the optimal control in a case a = go, i.e., 



<l>i.(x)=\xm 4i ? Jx), 

ajco 



(41) 



from Eq.(33). Since a=oo contain all mature ages individually 
fluctuated by internal stochasticity, we adopt this assumption to 
analysis of semelparous species in the next subsection. Addition- 
ally, it is known that the value function, Eq.(41), is the solution (in 
the viscosity sense) of another HJB equation: 



inf 

veV 



{[K + X]^(x)}+F(x) 



= 0, 



(42) 



[32]. Analysis of the optimal control which satisfies this equation is 
known as "stationary control problem" in control theory. This 
control is essentially distinct from one satisfied Eq.(37), because 
Eq.(42) does not depend on age. Consequently, the optimal state 
transition becomes the autonomous system as follows: 



dX{ =gj(X a ,v(X a ))da+Y t Hk(^a,y(^))dS k a 

k=\ ( 4i > 

X 0 =x. 

As Eq.(42) directly provides the value function, this case requires 
only two equations: Eqs.(34) and (42). 

Optimal life histories and HJB equations in semelparous 
and iteroparous species. As mentioned above, semelparous 
optimal control is categorized as stationary control problem on 
account of the random mature age. Due to the strong Markov 
property of the SDE, Eq.(43), the objective function of semelpar- 
ous species, lAsIx'M' can ^ e generally written as 



| AsI,x»M = 



da exp{ — Xa} E x [F s (X a ) S$ (a) 



(44) 



= E x [exp{-Xa*}<t>(X a *)S(a*)], 



we can simplify this equation if and only if d = 1 and the boudary 
is one point (e.g., S4 in File SI). Then, semelparous species have 
two optimal life schedule problems: (1) how to determine the 
optimal mature state and (2) how to control the growth rate until 
that states are reached. The former is known as the "Optimal 
stopping problem" in probability theory, as mentioned in [10]. To 
simplify the latter problem, we assume in this subsection that the 
optimal mature state exists and given. Setting the optimal mature 
age, a, a value function of semelparity becomes 
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nj/ sx (x) : = sup sup i/f s ^ t (x)= sup E x [exp{ -la}(j){Xa)S{a)]. 

v<eV x* veV 

(45) 

This equation suggests that the optimal control optimizes the 
expectation of survivorship until the mature age, a. In other words, 
the evolution of a semelparous life history implies the optimization 
of life span. 

From Eqs.(4) and (42), the semelparous value function satisfies 
the following Dirichlet boundary value problem of HJB equation: 

- inf{ [K + a}^ si (x)} +F s (x)=0. (46) 

When Eq.(46) has a unique solution in the viscosity sense, we can 
obtain an optimal control v*J(x), such that 



(47) 



from the theorem on p.228 in [31]. From the semelparous fertility 
function, Eq.(4), the value function satisfies 

\j/ s? Xx) = (j>(x) xeD. 

Therefore, the optimal control of semelparous species, Vs(x), (a 
Markovian control) becomes 



V S (Xa)=v(x,X a ) 



(48) 



As shown in the previous subsection, the optimal cotrol does not 
depend on the age. 

On the other hand, iteroparous species reproduce during their 
growth: the value function becomes 



poo 

i/r u (x) : = sup daexp{—Aa}E x [Fi(X a )Si(a) 

veV Jo 

= sup daexp{ — Xa}E x [Fi(X a )S(a)]. 
veV Jo 



(49) 



Therefore, the iteroparous value function is given by the solution 
in the viscosity sense of Eq.(37) and the characteristic equation 
Eq.(34). In other words, the evolution of iteroparous species 
implies the temporary optimization of the ERS which is different 
from the semelparous breeding system. Moreover, the maximum 
age a, provides a terminal condition to the ERS, which introduces 
complexity to the analysis of iteroparous species. 

Density of breeding age structure in fittest. We demon- 
strate, here, that this unification theory of OLSP and LDM can 
address the statistical characteristics of population structure 
associated with the optimal life history. We focus on statistics of 
breeding age; Oizumi and Takada showed objective function 
directly implied cumulant generating function (CGF) of that [10]. 
We then extend their consequence to our configuration. Let a° be 
breeding age. The CGF is written as a Taylor series of the 
logarithm of objective function with respect to — X as follows: 



& x {x) : = log^(x) 



k = 0 



k\ 



<«o>?>. 



(50) 



Probability density A(a), generating the CGF, is composed of 
basic reproductive number (known as 7£o) an d ERS by using 
Eq.(10) such that 



A(a) = ^- I dX exp{ Xa} ^ 
2m J i/> 0 



\j/ k (x) _ Ug(x) 

<J/ 0 (x) 



(51) 



If we can obtain the fittest 7^o, ERS, and cumulant generating 
function of breeding age directly from the value function, as for an 
objective function, analyzing the fittest population structure is 
useful. When one applies Eq.(29) to the cumulant generating 
function as an objective function in the optimal control, Eq.(29) 
should be a Laplace transform of the ERS. This means that the 
optimal control should not be a function of ). because expanding 
the logarithm of Eq.(29) into a Taylor series as Eq.(50) causes a loss 
of the meaning of the objective function in the optimal control. 
This holds true as long as maximizing the ERS is equivalent to 
maximizing fitness. The key point to calculate in the case is the 
second term of the RHS of another expression of the value 
function, Eq.(31): 



W>., a (x)-- 

F(x) 



inf E, 

veV 



dz{ H v x + 1} exp{ - Xx}F{X z )S(z) 



(52) 



obtained by integrating both sides of Eq.(37) for age a (Dynkin's 
formula [31]). In the iteroparous breeding system, we can show a 
condition that the value function can compose statistics of 
breeding age. Differentiating the integrant of the second term in 
Eq.(52) with respect to v, we obtain the condition that v* satisfies; 



E, [V,{Hl + A} exp{ -;.T}F I (jr t )5(T)] U„ 



,(53) 



E, [ V„ (n v *Fi(X % )) S(z) - {wf +X}F I (X % ) £ dsV vf i(Xs,V)S(T) ] =0 



where V v : = 2j=i 3/Sv,- an d F(X X )=F\(X X ). This equation 
illustrates that the optimal control does not depend on X if and 
only if the mortality does not include the control vector. In other 
words, deriving the optimal control maximizing the ERS from 
Eq.(52) without calculating the Euler-Lotka equation directiy is 
sufficient in this case. From a biological point of view, individuals 
of the species do not control their life spans in this case: mortality is 
not susceptible to mutation or strategy change, or is already 
optimized. In general, an optimal control provides the adaptive 
ERS and life span because altricity and precocity increase the 
fitness in r-selection [33,34] . In other words, v* being unconnected 
to X means excluding the control of precocity. According to the 
analysis of optimal control in semelparous breeding timing [10], 
precocity serves as a positive influence on persistence of the species 
under internal stochasticity. This consequence was explained by 
the following reason: the internal stochasticity yields individuals 
having precocity and late-maturing. The former individuals 
accelerate alternation of generation, namely, they contribute to 
increase their fitness. Since the others are exposed by higher risk of 
death, their contribution affects on the fitness and the 72-0 
negatively. Proportion of both individuals is given by their 
maturity, such as mature body size and fertility, and affects on 
their persistence. In contrast, empirical researchers reported that 
short-lived species had vulnerability for external stochasticity by 
sensitivity analysis in a geometric mean of each dominant 
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eigenvalue of interannual TMMs (long-term population growth 
rate) [8,35]. When species has control strategies excluding that of 
mortality, long-lived iteroparous species (a is large) should be 
adaptive because TZq is monotonically increasing in a. This 
consequence seems to consist with those reports. However, the 
other cases remain a distinct difference between the effect of 
internal and external stochasticities because the short-lived species 
can be adaptive under the former. Although this paper handles 
only internal stochasticity, there may exist a trade-off between 
both adaptations for stochasticities for evolution of long-lived and 
iteroparous species. When mortality is not controlled, the value 
functions can directly compose the cumulant generating function 
of breeding age in the fittest population structure such that 



®x(x) : = log^(x) 
-if 



k = 0 



(54) 



and 



(k) 



lim 



' 81 k 



®i(x) 



(55) 



By using the original age, the density of a® in the fittest population 
structure is given by the following function: 



lim 

j.-»o 



(56) 



where i/^M means TZo- 

In the next section, we examine several specific models in 
semelparous and iteroparous breeding systems and discuss their 
differences. 

Applications 

We show analysis of specific models in this section. Oizumi and 
Takada (2013) showed that internal stochasticity causes both 
increase and decrease of fitness [10]. Focusing on the latter case, 
the stochasticity is a kind of risk for species. We suppose that 
species have a choice to utilize two resources. Traits of these 
resources are different from each other, such that one of these 
gives the effect of "high risk and high return" and that the other 
gives the effect of "low risk and low return" on the individual size 
growth. As application of our theory, considering the optimal 
utilization of those resources, we show the effect of the utilization 
on the fitness. Accordingly, the species suitably averting risk 
maximizes the fitness in their habitat. 

Two-resource utilization model. We consider a simple size 
growth model as an application of our theory. When X a eR + is a 
body size at age a, we assume that the species can choose two 
kinds of resources: the species using a resouce, R\, have the 
following growth rate of size such that 



dX\ =y l Xlda+a l X\dB\ 



n 



-x, 



(57) 



and another resouce, R 2 , also being 




<S3 <S3 



Figure 1. A resource utilization problem. This figure illustrates 
how a fish wavers in its choice. R\ represents high risk but high 
expected growth rate, while both quantities are low in R2- These 
resources fluctuate independently of each other. In other words, the 
two resources provide individuals with different internal stochasticities. 
The fish should optimally choose the best utilization of both resources. 
doi:10.1371/journal.pone.0098746.g001 



dXl =y 2 X 2 a da + G 2 X 2 a dB 2 a 
Xl =x. 



(58) 



Thus, we assume that resources R\ and R 2 operate life histories of 
individuals via their growth rate of size. Let v a be utilization 
frequency which represents a frequency of utilization in both 
resources. The individual growth rate of size, then, follows 



dX a =[yi(l-Va)+y 2 v a ]X a da+ [o x {\-v a )dB x a +a 2 v a dBl]X a 

(50, 



We assume yieR+ >y 2 eU (y 2 could be negative), rjj >ff2>0 i.e., 
R\ -specialists (v o = 0) have higher risk and the expectation of 
growth rate than ^"Specialists^,, = 1). Conversely, ^-specialist 
has the other risk that the individuals have lower survivorship until 
mature age than R\ -specialists because of slower growth rate on 
the average. Therefore, individuals should find an optimal 
"utilization frequency" ve[0,l] to minimize the Hamiltonian with 
A under the environment (e.g., Fig. 1). 

Eq.(59) is well-known as typical "optimal portfolio selection 
problem" in mathematical finance and economics. Economists 
apply the problem to find an optimal investment of their wealth 
[13]. 

We assume a mortality in Eq.(5) such that 



where we set 



fi{y,v) = const, 



j«(j ; .v)= J u 0) 



(60) 



which does not have control. We are interested in difference of the 
optimal utilization v of Eq.(59) in different breeding systems. As 
preparation for analyzing the optimal utilization, the adjoint 
Hamiltonian of Eqs.(59) and (60) becomes 
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K = -[7^-v) + 72v]x-^-^[(a l (l-v)) 2 + (a 2 v) 2 ]x 2 ^+ f i 0 . 



(61) 



Let <p(x)eC 2 (M + ) provide an extreme value of \TC X + A] <p{x) with 
respect to v such that 



[K+X\<p(x)\ v=vf =o. 



(62) 



Then, the value satisfies 



v\ 9 (x))- 



■ + 



(yi-y 2 )^^W 



(63) 



and we obtain a nonlinear operator by substituting Eqs.(63) into 
(61) as follows: 



Hi (<p(x)):=[Hl + X\<p(x)\ v= j 



y x o\+y 1 o\ 



X £<P( X )- 



1 °M ,2 8 

2c\ + a\ dx* 



q>[x) 



+ i ^ ^-+(A«o + AMx) 



(64) 



( ff 2 + ff 2)_ Kx) 



P : 



log0(x) 

log | 



they showed the fitness by using p as follows: 



ox 2 . 



P+ Vi 



or 



-to- 



rn 



(69) 



It was shown that the stochasticity decreases the fitness if and only 
if p is less than one. In other words, the optimal utilization possibly 
exists at p< 1 in the model. If p is more than one, v of Eq.(59) is 
obviously equal to zero because the stochasticity affects the fitness 
positively. In persistent species, large p makes species advanta- 
geously at any parameters. 

Results 

Semelparous optimal resource utilization 

From Eq.(46), the value function satisfying Eqs.(4), (59), and (60) 
is generated by the following HJB equation: 

- inf{ [K + l]^ sx (x)}+F s (x)=0, (70) 

where <j)(x) : Z>clR + \(0,x)->(l,oo) in this case. Since Eq.(70) is a 
quadratic function of v (e.g., Eq.(61)), the extreme value is 

determined by (ij/sx{ x )j from Eq.(63). Substituting the value 

into Eq.(70), we obtain a nonlinear ODE from Eq.(64) as follows: 



Hereafter, we use frequently those equations, Eqs.(63) and (64), in 
our analysis. 

Reproductive timing in semelparous species. This sub- 
section describes results optimal reproductive timing in [10]. They 
used a case that Eqs.(57), (4) and (60) (or (58)). The objective 
function \j/ S)i (x) then staisfies 



■[H x + ^ S) (x) + F s (x)=0, 



where 7i% = 7i v Y \ n . The solution of this is 

x x i v = 0 



^W = (^) P <H**) 



(65) 



(66) 



H x {ij, sx (x))+Fs(x)=0. 



(71) 



To find the solution of Eq.(7 1), we assume a solution *jjs>.( x ) = Cx p 
(C#0) at x<x, and substitute it into Eq.(71) as follows: 



1 o 2 a 2 p(p-\) 1 (yi-y 2 fp 



2 a\ + al 2(a 2 + 4)(p-l 



-(H 0 + A) 



= 0. 



Setting p) to satisfy Eq.(72), we obtain 



X\ Pk 



{ l / sx( x ) = (~) H x ) 



and the constant, C, is given by 



Cx" 



(72) 



(73) 



Analysis of the above objective function showed that the optimal 
mature body-size x. satisfies 



log<K*) _ log<Kx*) 
logf x * log 3 - 



(67) 



It is convenient for analyzing Eq.(59) that the optimal mature age 
does not depend on parameters such as, y lt o\, and fi 0 . Therefore, 
the model has the same optimal body-size and age irrespective of 
the control parameter, v. Setting 



(j>(x)x-n 

to satisfy the boundary condition in Eq.(46), i.e., 
linVfi *l>si.( x ) = < t > ( x )- Eq.(73) can provide an unique optimal 
frequency v (e.g., the theorem on pp.228-232 in [32]) despite that 
px possibly have three different values, because the characteristic 
equation Eq.(9): 



^ Px <l>(x)= expi log0(x)-pjlog-f = 1, 



gives the following relationship 
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0 



/^-specialist 



Generalist 



/fi-specialist 



Generalist 



Index p 



Figure 2. Semelparous optimal utilization for each p. This figure 
shows the semelparous optimal resource utilization, Eq.(76), depending 
on p. Two distinct types of feeding habitat exist. A small or intermediate 

value of pe^0,l — \(f\— V2)J makes the species a generalist: the 

larger the value of p, the more the species favors risk and becomes 
specialist. Parameters are y, =0.16, y 2 = 0.1, <7i = 1.0, <j 2 = 0.12, and 
/j 0 = 0.01. 

doi:1 0.1 371 /journal.pone.0098746.g002 



P = Ph 



(74) 



from Eq.(68). Furthermore, Eqs.(72) and (74) can provide a unique 
fitness to be mentioned later. If p < 1 , we can find the optimal 
control from Eq.(63) because the value function, \j/ S)i (x), becomes 
a concave function. 

Substituting Eqs.(73) and (74) into Eq.(63), the optimal 
utilization, Vs, becomes 



Yi-72 



(o2 +0 2)(l-p) ! 



(75) 



This optimal utilization is constant, and it means that the species 
should conserve a frequency of resources during its lifetime. 
Regarding Vs as a function of p, p is an important index to 
determine the utilization. If the index is large, it shows that(76) 
semelparous species has a tendency toward risk appetite (e.g., 
Fig. 2) because i>s is small. Then, the optimal resource utilization 
continuously changes with respect to p. Since the continuity of 
optimal control with respect to p is provided by the second-order 
term of v in the diffusion term of the Hamiltonian, Eq.(61), it is 
different from the bang-bang controls appearing in deterministic 
models, which do not have second-order terms of control 
parameters. 

From Eqs.(69), (72), and (74), we obtain the fitness of the fittest 
as follows: 



i- °"i - , 
\P\^P + 



(»-**) 



?1°2- 

<x?- 



7l"\ 



l ofgj(l-p) 
2 <s]+al 



if v s (p) = 0 



(76) 



2(a] + al)(l-p) 



-Ho >f 0<v s (p)<l 




Index p 

Figure 3. Fitness of optimal utilization in semelparous species. 

We calculate the value of the fitness of semelparous species with 
respect to p between zero and 1 . The vertical dashed line represents the 
boundary between specialists and generalists as given by Eq.(77). The 
fitness is always a monotonically increasing function of p in the 
persistent region of the species Ws(p)>0). It is remarkable that a large 
p causes individuals to favor risk and increase their fitness. This figure 
shows the optimal resource utilization actually having advantages over 
specialists of each of the resources. Parameters are the same as in Fig. 2. 
doi:10.1371/journal.pone.0098746.g003 



from the RHS of Eq.(72) (e.g., Fig. 3). If ^s(P) is non-negative, the 
species is persistent. In that case, the fittest becomes Ri -specialist or 
generalist and ^-specialist is never selected because of magnitude 
correlation of each parameter in our definition. Nature selects 
generalist when p is within 



0<p<l- -4(vi-y 2 ). 



(77) 



otherwise Ri -specialist is selected. 

The dependence of utilization on p can be explained by a trade- 
off between the alternation of generation and TZq. It was shown in 
[10] that the internal stochasticity increases the number of 
precocious individuals, however, decreases that of mature 
individuals. A large value of p expresses a small difference 




!<>»[« 



Figure 4. Transition of the mature age distribution. Whenever p 
becomes large (pi<p 2 <h), the mature age distributions of every 
utilization behavior are shifted to younger ages than those under smaller 
p as in [10]. If the fittest species is a generalist, the change in distribution 
becomes extreme. Then, the age distribution is provided by S5 in File SI. 
We chose ij>{y) = Ry l l(x[+y l ) as the fertility rate function. This function 
has a unique p with respect to x, and the optimal body size, x, is obtained 
by calculating the derivative of (log[$(3c)]/ log[x/x]) =0 with respect to 
x. We substitute the RHS of that equality into x of the distribution. Then, 
we obtain this figure by changing x. Then, p becomes inversely 
proportional to x in this fertility function. Parameters are R = 10, x c = 1.0, 
/ = 2, x — 1 . 1, 1 .3, 1 .5, with the others being the same as in Fig. 3. 
doi:1 0.1 371 /journal.pone.0098746.g004 
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Gineralisl 



/f, -specialist 




[ikIcn fi 



Figure 5. Transition of the TZ a . This figure shows changes in the TZq 
with respect to p. The TZq of the fittest species increases as p increases; 
however, it decreases when p reaches the RHS of Eq.(77), which means 
that the growth strategy changes from the conservation of the TZg to a 
hasty alternation of generation time. Utilizing R\ usually, the TZo of the 
fittest is higher than that of v = 0. We simulate the TZq under the same 
parameters as in Fig. 4. To show the proportional connection between 
p and the TZq, we change the variable Uc-»3.5 —y) and simulate it within 
(0, 3.5) because p is in inverse proportion to x. Parameters are the same 
as in Fig. 4. 

doi:1 0.1 371 /journal.pone.0098746.g005 

between a mature body size and the initial body size, a large 
fertility <j)(x) or both, and makes the reproduction number of 
precocious individuals compensated for the decrease of number of 
mature individuals. Small p makes the opposite consequence that 
the reduction of the number of mature individuals exceeds the 
reproduction number of precocious individuals (e.g., Fig. 4 and 
Fig. 5). 

Iteroparous optimal utilization 

In this subsection, changing from the breeding system of Eq.(4) 
to Fi(y), we compare the optimal strategy of semelparity with that 
of iteroparity. We use the value function from Eqs.(31) and (49), 
and the same Hamiltonian as in Eq.(61). 

We assume the fertility function to scale with body-size 
allometric law, such as for the biomass of shoots and body size 
in trees [36], as follows: 



Fm(y) : =by' s (0</f<l), 



(78) 



where b and /? represent the fertility rate and an allometric 
exponent within the domain (0,1), respectively. Using the 
Hamiltonian, Eq.(61), we can obtain an optimal utilization of 
the iteroparous species identical to the one of the semelparous 
species. From Eqs.(37) and (63), the value function in Eq.(31), 
becomes a solution of the nonlinear PDE 



8a mAX 



(79) 



because 



H,(w la (x)) = mf{ [W x +X]w Ka {x)}, 
and the optimal utilization satisfies 

vi (x) = max { min { v 1 " ( w x , a (x) ) , 1 } ,0 } , 
from Eq.(63). To derive the solution of Eq.(79), we assume 



(82) 



(80) 



wx,a(x) = sxp{rj x (a-a)}bx l \ 
and substitute it into Eq.(79). Then, rj x becomes 



(81) 



-H 0 - A 



if v,(j8) = 0 



(82) 



, 1 (Vl -7l) 



a\ + a\ 2 a] + a 2 2 2 (<T? + af)(l -0) 



-H 0 -X if 0<V!(j9)<1 



Eq.(81) is guaranteed as an unique solution of Eq.(79) from the 
uniqueness of the viscosity solution [28]. Namely, r\i can be 
expressed by using the function of semelparous fitness, Eq.(76), 
such that 



(83) 



This suggests that the iteroparous optimal utilization has common 
and different characteristics compared to the semelparous one. We 
here use the expression, Eq.(83), to emphasize the common 
characteristics between semelparous and iteroparous optimal life 
histories. Substituting Eqs.(79) and (82) into Eq.(80), we obtain the 
optimal utilization as follows: 



\c\ + c\ (o? + o|)(l-/9 



(84) 



without having to solve the Euler-Lotka equation, as the mortality 
does not include control parameters. When we regard the 
utilization, Eq.(84), as a function of /?, the functional form 
resembles the optimal utilization of semelparous species, (Eq.(76)). 
Then, the exponent, serves as p in semelparous species. 

Using Eqs.(49), (81) and (82), the iteroparous value function in 
Eq.(49) becomes 



f" bx^ 

^uW = daexp{ri x (o(— a)}bx^= (expj^a} — 1). (85) 

Jo 1i 

Since Euler-Lotka equation, Eq.(9) being composed of Eq.(85) 
becomes transcendental, we cannot find the fitness of the 
iteroparous species explicitly. Then, we use the following 
inequality 



Ai(<x,j8)<;IsG8)+toc / ', (86) 
where the RHS of Eq.(86) is the dominant characteristic root of 



lim 4i nt 

cifoo 



bJ_ 



from the Euler-Lotka equation, Eq.(9), applying the "Basic 
optimal life schedule theorem" (S2 in File SI) to a. Since 
individuals in initial states normally contribute little to reproduc- 
tion in nature, we roughly assume that bx^ is sufficiently small. 

Considering the persistence of the species (Ai(a,/?) >0), Xs(fi) 
should be non-negative. In this model, the optimal utilization is 
supposed to maximize not only the fitness but also the ERS of the 
original age: 
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w 0 , a ( x ) = expjl s (0)a }W ae(0,a) (87) 

Additionally, if the fitness, 2.i(oc,fi), is a monotonically increasing 
function in /?, /? has an identical meaning to that of p in 
semelparous species. To prove this, we show that the ERS 
increases monotonically in /?. Since satisfies 



ls>0 y pe{Q\0<n(Q)<l}, 



and Eq.(69) is a monotonically increasing function of p in h* x >0, 
we conclude the proof. 

From Eqs.(56), (87), and the TZq: 



^ioW = ^r(exp{lsW4- 1 ). ( 88 ) 
the breeding age density becomes 



Is 0?) exp{! s (jS)a} 
A!(a) = J (exp|l s ()S)«}-l) 



W)>0 



As(«=0. 



(89) 



If As(/?) is positive, the age density shows that it skews toward older 
ages. In this case, the contribution of older individuals to 
reproduction is important for the persistence of the species. Even 
if As(/?) is equal to zero, the contribution is not negligible because 
the density has a uniform distribution. Since Eq.(76) maximizes 
the fitness of semelparous species, As(p), the optimal utilization of 
iteroparous species also maximizes As(/?). Although for large 
values of p and /? species in both breeding types favor more risky 
behavior, their breeding age structures are different, such that the 
semelparous mature age density is L-shaped and that the other is J- 
shaped (e.g., Fig. 4 and Eq.(89)). Consequently, the optimal 
utilization of iteroparous species enhances the contribution of 
older individuals to reproduction and differs from that of 
semelparous species because their longevity provides them with 
many opportunities for reproduction and with sufficient time to 
reach a large size. 

Considering the persistence of the species (\l/i 0 (x)>l) with 
respect to the maximum age, a, the following relation should be 
satisfied 



(PDE, TMM, and IPM) because it conserves continuity of states 
and does not require differentiablity of parameters: g(;) and ff(-). 
These parameters represent statistics of stage transition rate in 
TMM; they appear as components of Hamiltonian in the path- 
integral formulation. This Hamiltonian has a significant meaning 
in OLSP because the optimal control of life history minimizes or 
maximizes it depending on fertility function. There are two 
approaches in control theory that are to solve the HJB equation 
and maximum principle (MP). The analysis of optimal stochastic 
control using the HJB equation has been developed in various 
fields, including engineering and finance. However, many 
theoretical biologists commonly have used the MP approach in 
the analysis of OLSP [15,16,37,37]. The stochastic MP is proved 
by Peng (1990) [39], but it is not overwhelmingly popular in 
theoretical biology. According to Yong and Zhou ( 1 999) [31], both 
methods are formalized by the common Hamiltonian. There are 
two different viewpoints in those approaches. Idea of MP is 
originated from Hamilton's canonical system which is based on 
dynamics of a particle system in classical mechanics; i.e., OLSP 
based on MP describes an individual optimal life history. On the 
other hand, the idea of HJB equation is originated from 
Hamilton^Jacobi equation (PDE); it describes the optimal life 
history as optimizing cohort dynamics. These two approaches are 
equivalent in the sense that solutions of one can be represented by 
those of the other. Additionally, the correspondence of the HJB 
equation to the stochastic MP was shown via the idea of a viscosity 
solution by precursors [28-31,40]. They showed that the value 
function and its derivatives in the HJB equation correspond to the 
co-state variables in the stochastic MP. The Hamiltonian, which 
mathematical biologists use in the OLSP associated with MP 
approach, was merely one of the mathematical procedures for the 
analysis. The Hamiltonian now forms an important element of the 
demographic model. The path-integral formulation unifies 
stochastic control theory and LDMs via the Hamiltonian, and 
we showed that optimal strategies usually minimize 7i x + k. This 
Hamiltonian, Tt x , forms a counterpart of the Fokker-Planck 
Hamiltonian, referred to the "adjoint Hamiltonian" in Eq.(l 1). In 
physics, the Hamiltonian refers to the total energy of the system. 
Using this analogy, we can interpret Hamiltonian as a biological 
energy, whereby individuals consume energy throughout their 
lifetime. Then, the value function derived from the HJB equation 
represents the lowest energy consumption over the lifetime, and 
the Euler-Lotka equation converts the life history into population 
dynamics. Therefore, we can omit the analysis of population 
dynamics in the LDM because the theory in this study showed that 
the following equations provide the unification of OLSP and 
LDM: 



a> = log 

HP) 



1 + 



HP) 



(90) 



from Eq.(88). Since Eq.(85) increases monotonically in a and 
Eq.(86) is proved by using the basic optimal life schedule theorem, 
iteroparous fitness is a monotonically increasing function in a. 
Therefore, iteroparous species evolve to have optimal utilization 
and to survive as long as possible, as found for trees. 

Discussion 

This theory generalizes from the previous study [10] and 
extends to general OLSP in LDMs. The path-integral formulation 
is more suited to address controlled life history than the others 



^w la (x)-ig{[K + X\w x , a (x)}=0 
W),Jx)=F(x) 

ij/ A (x) = dawx, a {x) = \, 
Jo 

or another HJB equation: 

-mi{[K x + )^ x {x))+F{x)=Q 

if and only if the species does not have a terminal condition, such 
as maximum age, from Eq.(37). 

The two-resource utilization model shows that optimal strate- 
gies behave differently depending on breeding systems and fertility 
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functions, even if a species occurs in the same habitat. Then, the 
convexity of the objective function is the keyword of all of the 
optimal strategies analyses in this study. To explain the importance 
of the convexity, we introduce Jensen's inequality. Let f(y) be a 
concave function with respect to yeR, such as / (y) > 0 and 
/ (y) < 0. Then, the function satisfies the following inequality for 
an arbitrary random variable X a , 

E x \f{X a )]<f{E x [X a ]). 

Incidentally, as for the convex function, the inequality becomes an 
opposite magnitude correlation. As Jensen's inequality suggests, 
the exposure of a species to risk becomes advantageous when the 
objective function has high convexity. Since the convexity depends 
on life history, semelparous and iteroparous species possibly have 
different growth strategies even if they share several elements of 
life history, such as Eq.(l), and n{x,v) in the habitat. In this case, 
the difference in fertility yields different convexities of the objective 
function and strategies. In other words, the breeding system 
determines what the species optimize. 

In semelparous species, the parameter, p, represents an index of 
convexity degree in the value function. Although the previous 
study [10] did not mention the convexity, the positive effect of 
internal stochasticity is caused by the strength of the index. As 
Oizumi and Takada (2013) reported, there was a trade-off 
between precocity and altricity in the OLSP. The stochasticity 
yields both precocious and slow-growing individuals. The former 
makes alternation of generations faster, while the latter decreases 
TZq by increasing the risk of death. The paper showed that the 
index, p, determines the sensitivity of TZq in the stochasticity. 
Especially, a small p decreases IZo in the domain, pe(0,l). 
Consequently, species having a small p utilizes smaller risk, such as 
i?2- The optimal utilization, Vs, continuously decreases with p. 
Then, nature selects a species of specialists or of generalists in 
resources depending on p. 

The risk appetite of semelparous species depends on p, 
composed of the mature body size and </)(x) in the optimal 
utilization, while that of iteroparous species depends on the 
allometric exponent p. This has the same characteristic as the 
index p in the optimal utilization of semelparous species. If a 
species has a large value of /}, the fitness is also high. Those 
parameters characterize the convexity of the objective function. As 
Jensen's inequality shows, the convexity determines the effect of 
internal stochasticity on fitness. A large value of p or /? makes the 
objective function close to a convex function. When they have the 
same convexity, p = fi, semelparous and iteroparous species have 
the same risk appetite and the optimal utilization. 

In contrast, the breeding age distribution of iteroparous species 
is different from that of semelparous species. The trend in age 
distribution depends on Xs{fi). If ^s{P) is positive, the breeding 
age distribution skews toward older ages (e.g., Eq.(89)). Then, the 
persistence of the species is determined by the maximum age a. 
Therefore, long-lived individuals are important for population 
growth in this case. 

Another meaning of p and /?, from a biological point of view, 
represents the efficiency of conversion from adult body size to 
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